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I. Introduction 


As science advances, with the resulting advances in technology, we 
have new tools and new capabilities which influence our world in many 
ways. This new technology not only influences the traditional art forms 
but also produces new forms of art. The development of high speed elec- 
tronic components and circuits, the cathode ray tube, the video camera, and 
inexpensive video tape recorders enabled the development of video art. 
Advances in integrated circuit design and fabrication techniques have led 
to the development of small but powerful computer systems which can be 
utilized by the video artist to achieve a new dimension of control over 
the video image. With a computer-based video synthesizer (C3VS), one can 
generate a sequence of images while controlling each individual image with 
detail and precision that is many orders of magnitude greater than is 
possible with manual control. 

The ability to control the dynamics of the image is especially useful 
to the artist if the system is capable of generating the image in wee time. 
With this requirement in mind, the natural choice of devices for conver- 
ting electrical signals to visual images is the conventional video system. 
This choice also gives the capability of recording the video compositions 
with a conventional video tape recorcer anc of broadcasting to a large 
audience through existing network systems. 

There are basically two modes of operation of the system: inter- 
active-compositional mode and automatic-production mode. In the compo- 


sitional mode, the artist can enter programs ard parameters througn the 
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keyboard, observe the resulting sequence of images, and then modify para- 
meters through either the keyboard or a real time input and thus build 

up a data set for é complete piece. The data set, representing all the 
aesthetic decisions made by the artist, is stored in the computer at each 
stage of the composition. When the composition is finished the systen 
will operate in the automatic-production mode generating the final video 
sicnal in real time with no intervention by the artist. The artist may 
also choose to use a combination of these two modes in an interactive 
verformance or to allow an audience to interact with the system operating 
automatically. The system is structured so that all of these variations can 
be accomodated by appropriate programming. 

The system may be operated as a generating ssutneetees which produces 
a video signal entirely from internal signals or as a processing synthesizer 
which utilizes video signals of external origin such as a camera. Either 
of these two types of operations is carried out by a configuration of 
elements seaues. each of which performs a class of functions, with the 
specific function during one frame being determined Sy the control parameters 
recieved from the computer. 

Since the computer functions only to generate the parameters which 
govern the behavior of the synthesizer modules, a video signal will be 
generated sitnete Goeet oh of the computer. If the computer is stopped, 
the system will simply repeat the current frame until the parameters are 
changed. Thus the artist may choose to stop the computer and examine 
a single frame, or he may alter the program so that a tien sequence is 


displayed very slowly or repeated very rapidly. 
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II. General Design Considerations 


The NTSC video format is shown in figure 1. The time interval 
represented by one line is the time of one horizontal sweep or 1/15,734 
sec. = 63.5 microseconds. The number by each line indicates the number 
of times that each format is repeated. There are 483 active scan lines 
and forty-two lines of vertical blanking making a total of 525 lines to 
compose one frame (two fields). The first line (a) consists of the 
horizontal syne pulse, which is five microseconds long occuring during a 
ll microsecond. blanking interval, followed by color burst and the picture 
information which last 52.6 sieseeeeonee. The next line (b) shows the first 
equalization pulse which is followed by five more and the beginning of 
vertical syne (lines c,d). Vertical syne starts in the middle of the line 
for this field and at the beginning of the line for the next field (line m). 
Lines e,f,g, and h show the completion of vertical syne with serrations and 
six more equalization pulses. Vertical blanking is shown in lines i and }j. 
Tne second fiela-stancs in line j. Line k shows the 241 lines which inter- 
lace with tne first field to complete the frame. Finally there are 6 more 
equalization pulses before (line 1) and after (line n) vertical blanking 
(line m). Line o shows twelve more lines of vertical blanking. This sequence 
is repeated 30 times per second making a total of 108,990 frames der hour. — 

The portion of the video sisnal which carries intensity information 
is a continuously varying voltage and may be analyzed into components of 
different frequencies: low frequency components corresoond to course 


structures in the imafe, and hizh frequency components corresDond to fine 
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structures. Although the video signal may, in general, have an arbitrary 
shape, a monitor will reproduce an image which corresponds to only a 
limited frequency range. This range is called the bandwidth. Thus components 
of the video signal which are outside the bandwidth of the monitor will not 
be visible. In a system any device which limits the bandwidth of the video 
signal will degrade the fine structure or spatial resolution of the image. 

A video signal may be represented by a finite set of samples. These 
samples may be stored, eg. in a TBC or frame buffer, and then used to re- 
produce the video signal. In this process some of the information may be 
lost, but the lost information corresponds to high frequency components or 
fine structure of the image. If the sampling rate is sufficiently high, the 
lost inromation will be outside the bandwidth of the monitor, and no impair- 
ment of the image will occur. The minimum theoretical sampling rate which 
will retain the video information within a given bandwidth is called the 
Nyquist rate and is equal to twice the maximum frequency. Thus for broadcast 
quality, a minimum of 4.2 MHz. x 2 x 52 psec. = 437 samples per line are 
required. 

Each of the samples a be represented by a binary number with B digits. 

7 3B 
In general, a binary number with B digits has 2 discrete values, and a B-digit 
representation corresponds to - discrete gray levels. Since the video 
Signal may have any one of a continuous set of values, an error is introduced 
when it is represented by a discrete nunber. Fipure 2 snows the corre- 
sponéence between binary and decimal numbers, the correspondence between a 
continuous video signal and its discrete representation, and the resulting 


error. This error is called the quantization error, and its root-mean-square 
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value is given by cyt tL tea = Jt =m 
i,(2 = ) iz ak 
If B is sufficiently large, the spacing between the discrete gray levels 
will be small compared to the intrinsic noise of the system and will not 
be visible on a monitor. The minimum acceptable value of B may be estimated 
by considering the quantization signal-to-noise ratio which is given by 
S/N = 20 log \ {12° - = (6.02 B + 10.79) db. 

The signal-to-noise sae for a one-half inch VTR is about 40 db, so about 
five binary digits are required for a comparable quality (monochrome) image. 

For the analysis of equipment requirements for generating a video signal, 
we may utilize the measure of information given by the mathenatical theory of 
communication. If a message occurs with probability P, then the amount of 


information (measured in bits) is 


I=log (1/P), 
2 


Thus, the answer to a question which can be answered by yes or no with 
equal probability carries the quantity of information of 


log 1/(1/2) = log 2 = 1 bit, 
2 2 


Since a binary digit has two possible values, zero cae one, each may carry 

at most one bit of iaeomaaeton. . Thus there are 5 x 437 x 483 = 1,955,355 
bits of information in one pane, Ff monochrome video signal (assuming thirty- 
two gray levels and 4.2 MHz. bandwidth). For generating a color sirnal, one 
may represent each of the red, green, and blue primary signals with a five 
bit word specified at time intervals of 100 nSec; this corresponds to an 


information rate of 150,000,000 bits per second. Other exandles of amounts 
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of information are typewriter keystroke, about six bits, and one typewritten 
page (double spaced), 2,900 bits. The rate at which the human brain can 
process information has been estimated to be about forty bits per second. 

The mathematical theory of communication introduces another concept, 
redundancy, which is useful in analyzing a video synthesizer system. A 
message which contains N bits of information may be coded in a way thet uses 
more than N binary digits. For example, the message pair (yesyno) may 
be coded as (111/000) using three binary digits instead of one per message. 
In this case the coding is redundant. The pattern shown in figure 3 consists 
of 16 x 16 = 256 squares, each of which is either all black or all white. 
Thus byuneprecanems wnite Dy one and black by zero, any pattern of this 
Format can be represented by a code consistinz of 256 sina dinits. If all 
possible patterns are allowed, then one pattern carries 256 bits of in- 
formation. A circuit which generates a video signal corresponding to this 
pattern codes the message into a form with 1,055,355 binary digits (assuming 
the spatial and intensity resolution as anova). The redundancy of the video 
Signal is further increased if the set of possible patterns is restricted 
by requiring that the total vattern is built of sixteen 4 x 4 dlocks each 
being one of the followinz: all black, all white, black and white as in 
the upper left pana corner, or the latter with Dlack and white interchanged. - 
Then two binary digits coce ie Ghotee for eacn block, and since there are 
Sixteen Dlocks, thirty-two bits of information are contained in the entire 
pattern. 

The disparity between the rate at which the human brain can process 
information, wnicn limits the rate that an artist can mManidulate controls 


of a video synthesizer, and the rate that information must be senerated in 
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order to produce a video signal clearly shows that the synthesizer must 
be structured in such a way to exploit redundancy in the video signal. A 
microprocessor typically requires 2 to 10 MSec. to execute a single in- 
struction; thus it cannot possibly be used to generate a point every 

109 nS. as required for a video signal in real time. On the other hand, 
one video field last 1/60 = 16.67 mS., a duration in which several thou- 
sand instructions can be executed. Thus a microprocessor is capable 

of generating signals according to a complex algorithm | utilizing infor- 
mation supplied by the artist to produce control signals for high-speed 
special-purpose devices at a field-by-field rate. 


Thus we are led to the alerarchical structure shown in Ficure 4 


Typical channel capacities are shown for eacn interconnection. The 
synthesizer consists of hich speed special purpose circuits which 

generate a video signal with a character determined by the control 

Signals supplied by the computer. The control Signals fix the behavior 
of these circuits for an entire field. The computer takes information 
supplied by the artist, information defining the composition as a whole, 
and from it determines the control parameters required by the synthesizer 
for each field. Thus a gradual change in some picture parameters can 

be specified by the artist by a small set of numbers. In the simplest case 
only two numbers are required, the frame count of the first and last frame 
of the sequence. The conputer utilizes this infornation to calculate the 
corresponding picture parameters for eacn field; thus it may preduce 


several thousand control values. 


eS ee ear Fad 





III. System Structure 


The CBVS consists of two parts: the computer section shown in the 
lower section of figure 5 and the video section shown in the upper section 
of the ficure. 30th sections operate simultaneously and independently, 
communicating through the buffer memory which has a capacity of 1,924 

ixteen-bit words. Eacn of these words is either a picture element, a 
number which controls some function of the video section and determines 
some aspect of one field of the video imase, or it is a picture feature, a 
number determinea by the video section and may depend on an external 
signal such as a video camera signal. The buffer memory is connected 
to the computer bus through a sixteen-bit parallel interface which is 
structured in such a way that each word in the buffer memory is addressable 
and may be read or written in exactly the same way as words in the main 
computer memory. This memory-mapped I/O system Simplifies the software 
which controls the buffer memory. In order to update an element such 
as a control D/A, the computer must execute an instruction which stores 
+he new value in the location corresponding to that element. 

Durins the active scan time, tne control computer reads features 
from the buffer memory and generates elements for the followine field 
ana stores ‘en in the buffer: memory. During the vertical Dlanking ineepvar: 
infornation is transZerred througa the element Dus from the buffer menory 
to the element modules or from the feature nocules to the suffer menory. 
Tne desiznation of a particular area of the buffer memory as an element 


or feature is under vropran control. during the transfer between tne 





suffer memory and the element bus, the computer is locked out of the buffer 
memory. %n completion of the transfer, the interface generates a vectorec 
atene which requests the computer to zenerate parameters for the next 
field. 

The computer system consists of a DEC LSI-11 microprocessor which 
has a sixteen-bit word length and an instruction execution time of about 7 
microseconds, a Teletype Keyboard and printer connected through a serial 
interface, 20 K of dynamic memory, and a dual floppy disk system with a 
capacity of 256,256 bytes per diskette. An additional serial interface 
is also available for connecting through a modem to other computer systems. 
Tne entire system is dedicated to the synthesizer system. 

Tne overall timins is determined by a 9.7552434 Hz clock which is 
phase locked to the subcarrier (3.579545 MHz). This frequency is chosen 
to insure a coherent subcarrier and to divide the active portion of the 
scan line into 512 vixels. The red, green, and blue Sicnals are generated 
independently, and the chroma encoding is done with analog circuits; thus 
there is no advantage to following the common practice of making the pixel 
rate an integer multiple of the subcarrier frequency. With this clock 
frequency, a full nine bit word is used to define the horizontal Soestion 
on the active portion of the raster. Fisure 6 shows the X and Y wave forms. 
The X-Y module zenerates twenty bits of timing information (ten bits for 
horizontal, including the blanking period, and ten for the line count). 
Tnis module also generates sync, drive, burst flagz,and tne transfer recuest 
TR siznal which controls the timing of the buffer menory. 


Timing details of the interface and buffer memory ere shown in fisure 





7. The transfer request TR goes low at the beginning of vertical blanking 
initiating an arbitration for access to the buffer memory. If the computer 
is accessing the buffer memory, the current bus cycle is completed, then 
READY goes high, and the buffer memory controller cycles througn menory 
making the required elenent and feature transfers through the element bus. 
Wnen this is completed, READY goes low, control of the memory is returned 
to the computer, and an interrupt is generated requestine data for the 
following field. As indicated in the diagram, during the th field the 
computer is generating data for the N+ith field. 

The timings of the signals on the element bus are indicated in Fizure 
8. During the transfer, the memory controller generates the addresses. 
Ag-Agq » the clock signals, p » and é , and the status signals CME 
incicating a transfer from the memory to an element and CFM indicating 
@ transfer from a feature module to the memory. The signals ETF and FTE 
are generated by the synthesizer modules and initiate a controller Flement/ 
Feature mode change. The three phase clock system is used to control 
modules which have the structure shown in figure 9. Functions which use 
cata from the computer during the vertical Dlanking interval are disabled 
when the buffer memory accesses that particular element by a signal generated 
using é - This allows access to the buffer memory durins é - The third 
clock ¢ generates a memory write signal. | 

Time delays in the digital processing modules could produce errors 
and shifts of the imaze to the rignt. This is prevented by deskewing the 
output of eaca with a latch clocked by the master clock (2.755 meh a 


Compensation for the resulting 102.5 nSec. delay in each module is proviéed 
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by starting the X count at the besinning of the horizontal blanking interval 
ratner than at the end. An additional shift to the right or left is then 
achieved ay adding (mod 512) a constant supplied by the comouter. The 


default value of this constant is 404 + number of elements. 
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IV. Element and Feature Modules 


The structure described above supports a variety of element and feature 
modules which may be chosen and configured according to the taste of the 
artist. Our experience indicates that a large amount of work can be 
produced with a relatively small number of elements in a standard confisu- 
ration. Whenever possible a new element added to the system is configured 
in such a way that if the control word is set equal to zero it has no 
effect on the system. Thus a minimum amount of reprogramming is required 
following system expansion. 

Two general classes of modules have been developed, digital and hybrid. 
The hybrid elements are high-speed disital-to-analos converters used for 
generating the red, green, and blue video signals which are converted to 
NTSC format in the standard way and low-speed D/A converters used for gen- 
erating control voltages, field-by-field controllable, used to operate existing 
voltage-controlled analog image vrocessine systems sucn as keyers, raster 
manipulaters, etc. Another hybrid element is the analog video switching 
matrix. Four bits of one control word are used to select one of sixteen 
inputs for one output. 

Digital processing elements include: constant, a + constant, Y + ; 
constant, twelve-channel eteenciine demultiplexer with outpbut complement, 
and four-cnannel four-bit by sixty-four word memory. 

One of the possible ways of interconnecting ¢izital elenent nodules 
is shown schenetically in ficure 10. With this arrangement the X and 


Y siztnals are processed by the three Sequences (A,4 4,48) 


Vo 23g ? (3,5,8,3)), 
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and (C,C,C C.) to produce three different patterns or textures. These 
three sirnals are then combined by element D to produce a conoosite inarfe. 
Finally, the digital-to-analog converter produces an analog video signal. 
Tne form of the video signal generated by this system depends on both 
the choice of conficuration of modules and on the control parameters 
supplied by the computer. Since the control parameters are constant 
during one frame, the video signal may be represented by an equation of 
the — 

ve) = £] x(a), YUE), Ey(t)jeeeeeeeeEg(t) | 
where V(t) is the video signal which varies with time t. The structure 
of V(t) and of the resulting image is determined by the function f which 
is defined by the configuration of element modules. The time dependence 
of the video sicnal 3g shown explicitly; during one field, only X and Y 
change while the control values E,(t), Fj(t), E,(t),....E,(t) are held 
constant. The time devendence of the video signal has been divided into 
two classes: firstly, the variation from one field and the next is . 
determined >by the computer through the element values, and secondly, the 
variation during one Field is determined by the element modules through 


the X and Y signals for fixed element values. 
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VY. System Operation 


Tne power and versatility of this system may be seen by considering 
an application to an extrenely simplified version of the system consisting 
of a pair of camera signals which are mixed by a voltaze controlled mixer. 
With this system, only one elenent, a low-speed disital-to-analoz converter, 
utilizes the elenent pus, anc it generates a control voltage which determines 
the mixer operation for each frame. A flow chart of a program and an 
example of a data set are shown in figure 11. To use this system, the 
artist only needs to specify the nunerical values in the data set. 

Tne data set in this example corresponds to the mixer selectins the 
First camera until frame number 600, then fadine to the second camera 
until frame 700, holding the second camera until frame 2000, then fadins 
Sack to the first camera by frame number 2200. The values § and 4 in the 
parameter column cetermine the rate at which the fade takes place, and 
the numbers in the service routine column label indicate the selected 
service routine: a1 for no chanze, a 2 for increasing the control voltage 
Dy P units,;and a 3 for decreasing the control voltage by P units. 

After the artist nas stored these twelve numbders in the computer 
memory, the program may be started, and the computer will initialize 
the system and fo to a backsround orogram where it waits for an interrupt. 
After every otner interrupt the frame count is incre2asec by one and 
compared with the Frame count entry in the lata set. Then the approsriate 
service routine (S7) is selected and the element value is increased or 


decreased as required. Finally, the computer returns to the Sacxrrouné 





program and waits for the next interrupt. This process is repeated 
for every field thus generating the sequence of control voltages anc 
fadinz from one camera to the otner. 
The artist can observe the resulting sequence of images and then 
make changes in the data set to achieve the cesired result. This 
technique may be extended to more complex systens involving several 
elenents and feature modules with corresponding programs and data sets. 
Taus the artist can produce complex sequences with precise control of 
each frame. — when the composition is finished, the system will automatically 
cenerate the video signal which may be displayed on a monitor or recorded 


ona VTR., 
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